April 2020

Background

  • These slides show a proposed design of a Phase 2 trial with three experimental arms and no control

  • The design was not used, so these slides were instead used as introduction to graphical hypothesis testing

  • References to specific treatment regimens and disease setting have been removed

General

  • Three arms: TrtA, TrtA + TrtB, TrtA + TrtC

    • Refer to these as mono, combo 1, and combo 2
  • No control arm

  • Planned \(N = 75\) patients per arm, total of \(225\) patients

    • may be reduced to \(N = 40\) per arm
  • Futility analyis after 75 total patients have undergone sufficient follow-up time

Historical Control

  • Meta analysis suggests a \(20\%\) objective response rate for patients on current standard of care

  • ORR in each arm must have a confidence interval which lies above \(20\%\)

    • Equivalently, the ORR is tested with \(H_0: \pi_i = 20\%\) (exact binomial test, z-test, etc.)

Demonstrating Efficacy

A treatment must:

  1. Improve expected ORR of 20% on standard of care

  2. A combo must have significantly better ORR than monotherapy in addition to having better ORR than standard of care

First test ORR in each arm separately against 20% historical benchmark. Then test a combo against mono only if that combo better than standard of care.

  • Tests of superiority relative to mono are gated behind tests against historical control

Testing Strategy

Monotherapy arm must have:

  1. ORR statistically significant relative to historical benchmark of 20%

Combination therapy arm must have:

  1. ORR statistically significant relative to historical benchmark of 20%

  2. ORR statistically significant versus monotherapy

Graphical Testing: Overview

  • Graphical testing procedure let you prioritize hypothesis tests using a straightforward, visual diagram while maintaining the FWER at \(\alpha\)

  • You have \(M\) hypothesis tests, \(H_1, \dots, H_M\)

  • Assign local significance levels, \(\alpha_1, \dots, \alpha_M\) to hypothesis test such that \(\sum_{j = 1}^{M} \alpha_j = \alpha\)

  • When you reject a hypothesis test, \(H_j\), you can distribute its \(\alpha_j\) to other tests

    • You can reject these specific hypotheses at lower significance thresholds (Ex: \(0.025 \rightarrow 0.05\))

Graphical Testing: Diagram

  • Tests against historical control are given weight with \(\alpha / 3\)

  • Tests of combo against mono are assigned initial weight of 0

  • Promising arms can “give” \(\alpha\) to other arms.

Graphical Testing: Diagram

Graphical Testing: R GUI

Graphical Testing: Demo

  • R can open a GUI to manually construct graphical testing diagrams
library("gMCP")
graphGUI()

Graphical Testing: R code

alpha  = 0.025
# weights for each hypothesis tests
weights <- alpha * c(1/3, 1/3, 1/3, 0,  0)
# Matrix of edges, shows how alpha can be shared
m <- rbind(c(0, 0.5, 0.5, 0, 0),
           c(0, 0, 0, 1, 0),
           c(0, 0, 0, 0, 1),
           c(1, 0, 0, 0, 0),
           c(1, 0, 0, 0, 0))
# Make a graph object
graph <- new("graphMCP", m = m, weights = weights)
# Run the graphical test 
test = gMCP(graph = graph, 
            pvalues = p_vals, #raw p-values from hypothesis test
            alpha = 0.025)

Simulations

  • Use simulations to see operating characteristics of our study

  • See benefits of graphical tests:

    • Strict Type I error control with many tests
    • Increased power

Simulation Settings

  • Objective response for arm \(j\) is drawn from \(\text{Binom}(n, \pi_j)\).

  • Monotherapy success probability is varied from \(\pi_M = 0.20\) to \(\pi_M = 0.50\) in increments of 0.05.

  • Combo 1 ORR success set \(\pi_{C1} = \pi_M + C\), where \(C = 0, 0.05, 0.10, \dots, 0.40\).

  • \(\pi_{C1} \in [0.20, 0.90]\), depending on \(\pi_M\)

  • For simplicity, combo 2 ORR success probability fixed at \(\pi_{C2} = \pi_M\) for all simulations.

Type I Error Control

  • Assume a familywise Type I error rate of \(\alpha = 0.025\) for all 5 tests of ORR.

  • The testing procedure controls the Type I error rate well below \(\alpha = 0.025\)

Sample Size Level
40 0.023
50 0.017
60 0.013
75 0.013
100 0.016

Tests Against Historical Control

  • ORR is tested against against a point null at 20% using an exact binomial test.

  • The family-wise Type I Error Rate (\(\alpha\)) used in all simulations is \(0.025\)

  • Tests in R are conducted with:

binom.test(
  x = num_OR, 
  n = 75,
  p = 0.20,
  alternative = "greater"
)

Rejection Regions: Reject 20% Historical Control

Rejection Regions: Tests Against Historical Control

Sample Size Required OR Required ORR
40 15 0.375
60 21 0.350
75 25 0.333
100 31 0.310

Historical Control: Recycling Alpha

  • Require one less response for each successful arms
    • \(\alpha\) is redistributed (“shared”) across arms

Historical Control: Recycling Alpha

Sample Size Type I Error Required OR
75 0.025 23
75 2/3 * 0.025 24
75 1/3 * 0.025 25

Power: Superior to Historical Control

Mono and Combo Power Relationship

  • Mono can be retested if the combo tests are rejected,
  • The power of rejecting \(H_0: \pi_M \leq 20\) depends on \(\pi_C\)

Combo Superior to Mono

  • Power of rejecting \(H_0: \pi_{C_1} \leq \pi_{M}\).

Combo Superior to Mono

  • We may be concerned about isolation of effect if we don’t have a significant monotherapy

    • Ex: How do you know that TrtA isn’t driving efficacy alone?
  • We want to show that monotherapy is efficacious and that the combos are more efficacious.

  • Test this with the composite test: \(H_0: \pi_M \leq 20\%\) and \(H_0: \pi_{C1} \leq \pi_M\)

    • Test of efficacious monotherapy and combo superior to monotherapy

Combo Superior to Mono